Anomaly detection apparatus, anomaly detection method, and computer-readable recording medium

ABSTRACT

An anomaly detection apparatus 100 includes an image transformation unit 103 that calculates an image transformation parameter, based on an inspection image in which an inspection object appears, a reference image indicating a normal state of the inspection object and a parameter for image transformation parameter calculation, and performs image transformation on the inspection image using the image transformation parameter, an image change detection unit 104 that collates the reference image and the image-transformed inspection image using a change detection parameter, and calculates an anomaly certainty factor indicating whether there is a change in a specific region of the inspection image, a change detection parameter learning unit 106 that learns the change detection parameter, based on a difference between a training image indicating a correct answer value of the change and the anomaly certainty factor, and an image transformation parameter learning unit 108 that learns the parameter for image transformation parameter calculation, based on a collection amount derived from the difference between the training image and the anomaly certainty factor and to be applied to the inspection image that has undergone image transformation.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is a National Stage of International Application No.PCT/JP2017/035746 filed Sep. 29, 2017, the disclosure of which isincorporated herein in its entirety by reference.

TECHNICAL FIELD

The present invention relates to an anomaly detection apparatus and auanomaly detection method that are for detecting anomalies in an objectby collating a reference image and an inspection image that are shot ofthe object, and furthermore relates to a computer-readable recordingmedium that includes a program recorded thereon for realizing theapparatus and method.

BACKGROUND ART

Heretofore, apparatuses whose object is to automatically detectanomalies that are evident from the outward appearance of industrialcomponents, infrastructural structures and the like using images havebeen proposed. Such apparatuses map a reference image shot of the objectin a normal state and an inspection image shot of the object at the timeof inspection, and detect anomalies from the difference therebetween.Also, such apparatuses execute anomaly detection after having performedposition transformation (alignment), in the case where the position ofthe object in the inspection image differs from the position of theobject in the reference image (e.g., refer to Patent Document 1).

Specifically, Patent Document 1 discloses a defect detection apparatusthat detects defects using a reference image (model image) and aninspection image (input image). The defect detection apparatus disclosedin Patent Document 1 maps the reference image and the inspection image,based on feature points that are respectively extracted therefrom,transforms the coordinate systems of both images into a commoncoordinate system, and thereafter detects defects by deriving thedifference between both images.

Also, although not a technology for detecting anomalies, Patent Document2 discloses a technology for aligning two images. With the technologydisclosed in Patent Document 2, alignment is performed by extractingedges from a reference image (first image) and an inspection image(second image), and estimating position transformation such that thefeature amounts of the edges extracted from the respective images aresimilar to each other.

Incidentally, with the apparatus disclosed in Patent Document 1 or thetechnology disclosed in Patent Document 2, in the case where a largedifference caused by an anomaly or a large change in luminance arisesbetween the two images, a situation could possibly arise in which localfeature amounts of one image with respect to feature points of the otherimage do not coincide at corresponding points. In the abovementionedcase, a situation could also possibly arise in which the correspondencerelationship of pixels from one image to the other image is more complexthan the level that can be represented by affine transformation orhomography, such as a situation where the correspondence relationship isrepresented by a nonlinear transformation, for example. Alignment isdifficult in the case where these situations arise.

Accordingly, in anomaly detection using images, it is conceivablydesirable to enable alignment by feature points that is disclosed inPatent Document 1 and alignment by edges that is disclosed in PatentDocument 2 to be used in combination.

However, it is actually difficult to use the alignment that is disclosedin Patent Document 1 in combination with the alignment that is disclosedin Patent Document 2. This is because the evaluation criteria of whetherimages coincide are different in both, and the alignment accuracy cannotbe enhanced simply by using the features of both in combination. Thereasons for this will be specifically described below.

First, with alignment that is based on feature points, generally, aftermapping feature points between two images, an equation with a parameterfor alignment as a variable is created for every group of correspondingpoints that correspond, and the parameter for alignment is analyticallyderived by solving a simultaneous equation by a least-squares method orthe like. The solution for a homography matrix using the DLT algorithmthat is described in Non-Patent Literature 1 is given as a typicalexample.

On the other hand, with alignment that is based on edges, thetransformation parameter cannot be derived with a solution that solves asimultaneous equation. Therefore, with alignment that is based on edges,as described in paragraph [0018] of Patent Document 2, it is necessaryfor an edge image extracted from one of the images to actually bedeformed and superimposed on an edge image of the other image, and for atransformation parameter that reduces the difference between the edgeimages to be derived through trial and error.

In this way, the alignment that is disclosed in Patent Document 1 andthe alignment that is disclosed in Patent Document 2 employ completelydifferent methods of estimating the transformation parameter, and it isdifficult to extend these techniques to perform alignment that uses bothfeature points and edges.

In contrast, Non-Patent Literature 2 discloses an image transformationmethod that derives position transformation that reduces the differencebetween the image feature amount of a reference image and the imagefeature amount of an inspection image, and transforms the coordinatesystem of each image into a common coordinate system, based on thederived position transformation. Also, with the image transformationmethod disclosed in Non-Patent Literature 2, various feature amountseffective in image recognition can be learned in advance, without beinglimited to feature points or edges, and the coordinate systems can betransformed using the learned feature amounts. Thus, with the techniquedisclosed in Non-Patent Literature 2, there is a possibility of beingable to perform alignment that uses various features effective in imagerecognition such as feature points and edges in combination.

LIST OF RELATED ART DOCUMENTS Patent Document

Patent Document 1: Japanese Patent Laid-Open Publication No. 2012-032370

Patent Document 2: Japanese Patent Laid-Open Publication No. 2014-126445

Non-Patent Document

Non-Patent Document 1: Richard Hartley, Andrew Zisserman, “Multiple ViewGeometry in Computer Vision Second Edition” [online], CambridgeUniversity Press, 2004, p. 91, Algorithm 4.1, Internet <URL:.

http://cvrs.whu.edu.cn/downloads/ebooks/Multiple % 20View % 20Geometry %20in % 20Computer % 20Vision % 20(Second % 20Edition).pdf>

Non-Patent Document 2: Angjoo Kanazawa, David W. Jacobs, ManmohanChandraker, “WarpNet: Weakly Supervised Matching for Single-viewReconstruction” [online], University of Maryland, College Park, 20 Jun.2016, Internet <URL: https://arxiv.org/abs/1604.05592>

SUMMARY OF INVENTION Problems to be Solved by the Invention

Here, an anomaly detection apparatus that detects anomalies isenvisaged, by combining the alignment technology that is shown inNon-Patent Literature 2 and a discriminator that has learned to detectanomalies from an image pair aligned in advance, using an existingpattern recognition technology that is known today. Note that a supportvector machine, deep learning and Ada-boost are given as examples ofexisting pattern recognition technologies. According to such an anomalydetection apparatus, there is a possibility of being able to utilizevarious features in alignment.

However, since the features that can be utilized are not limited tofeatures suitable for anomaly detection, it may not be possible toperform highly accurate anomaly detection, even with such an anomalydetection apparatus. This is because the transformation parameter inNon-Patent Literature 2 is learned such that points that intrinsicallycorrespond are mapped, but features suitable for anomaly detection arenot optimized. In other words, pixels that do not correspond to eachother are included in a pair of images that include an anomalous place,but when the transformation parameter is merely learned such that pointsthat intrinsically correspond are mapped, there is uncertainty about howto map pixels that do not correspond to each other.

For example, there is a possibility of superfluous image transformationthat conceals anomalous places being executed, in which case, anomalydetection will be difficult. Conversely, there is also a possibility oferroneous mapping caused by a difference in luminance values beingperformed on an inspection image that is merely hit by light differentlyand is not anomalous. Such a situation arises from informationindicating whether a difference between the inspection image and thereference image is due to an anomaly that is desirably detected notbeing used in learning the parameter for alignment.

An example object of the invention is to provide an anomaly detectionapparatus, an anomaly detection method and a computer-readable recordingmedium that enable highly accurate anomaly detection, by simultaneouslyoptimizing a parameter for image transformation suitable for anomalydetection and a parameter for identifying whether an image that hasundergone image transformation is anomalous or normal.

Means for Solving the Problems

An anomaly detection apparatus according to an example aspect of theinvention is for detecting an anomaly in an inspection object, using aninspection image in which the inspection object appears and a referenceimage showing a normal state of the inspection object, the apparatusincluding:

an image transformation unit configured to calculate an imagetransformation parameter, based on the inspection image, the referenceimage and a parameter for image transformation parameter calculation,and perform image transformation on the inspection image, using thecalculated image transformation parameter, such that the inspectionobject in the inspection image overlaps with the inspection object inthe reference image;

an image change detection unit configured to collate the reference imageand the inspection image that has undergone image transformation, usinga change detection parameter, and calculate an anomaly certainty factorindicating whether there is a change in a specific region of theinspection image;

a change detection parameter learning unit configured to learn thechange detection parameter, based on a difference between a trainingimage indicating a correct answer value of the change in the specificregion and the anomaly certainty factor calculated by the image changedetection unit; and

an image transformation parameter learning unit configured to learn theparameter for image transformation parameter calculation, based on acorrection amount derived from the difference between the training imageand the anomaly certainty factor calculated by the image changedetection unit and to be applied to the inspection image that hasundergone image transformation.

Also, an anomaly detection method according to an example aspect of theinvention is for detecting an anomaly in an inspection object, using aninspection image in which the inspection object appears and a referenceimage showing a normal state of the inspection object, the methodincluding:

(a) a step of calculating an image transformation parameter, based onthe inspection image, the reference image and a parameter for imagetransformation parameter calculation, and performing imagetransformation on the inspection image, using the calculated imagetransformation parameter, such that the inspection object in theinspection image overlaps with the inspection object in the referenceimage;

(b) a step of collating the reference image and the inspection imagethat has undergone image transformation, using a change detectionparameter, and calculating an anomaly certainty factor indicatingwhether there is a change in a specific region of the inspection image;

(c) a step of learning the change detection parameter, based on adifference between a training image indicating a correct answer value ofthe change in the specific region and the anomaly certainty factorcalculated in the (b) step; and

(d) a step of learning the parameter for image transformation parametercalculation, based on a correction amount derived from the differencebetween the training image and the anomaly certainty factor calculatedin the (b) step and to be applied to the inspection image that hasundergone image transformation.

Furthermore, a computer-readable recording medium according to anexample aspect of the invention includes a program recorded thereon fordetecting, by computer, an anomaly in an inspection object, using aninspection image in which the inspection object appears and a referenceimage showing a normal state of the inspection object, the programincluding instructions that cause the computer to carry out:

(a) a step of calculating an image transformation parameter, based onthe inspection image, the reference image and a parameter for imagetransformation parameter calculation, and performing imagetransformation on the inspection image, using the calculated imagetransformation parameter, such that the inspection object in theinspection image overlaps with the inspection object in the referenceimage;

(b) a step of collating the reference image and the inspection imagethat has undergone image transformation, using a change detectionparameter, and calculating an anomaly certainty factor indicatingwhether there is a change in a specific region of the inspection image;

(c) a step of learning the change detection parameter, based on adifference between a training image indicating a correct answer value ofthe change in the specific region and the anomaly certainty factorcalculated in the (b) step; and

(d) a step of learning the parameter for image transformation parametercalculation, based on a correction amount derived from the differencebetween the training image and the anomaly certainty factor calculatedin the (b) step and to be applied to the inspection image that hasundergone image transformation.

Advantageous Effects of the Invention

According to the invention as described above, highly accurate anomalydetection becomes possible, by simultaneously optimizing a parameter forimage transformation suitable for anomaly detection and a parameter foridentifying whether an image that has undergone image transformation isanomalous or normal.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram showing a schematic configuration of ananomaly detection apparatus in a first example embodiment of theinvention.

FIG. 2 is a block diagram showing a specific configuration of theanomaly detection apparatus according to the first example embodiment ofthe invention.

FIG. 3 is a diagram showing an example of an inspection image that isused in example embodiments of the invention.

FIG. 4 is a diagram showing an example of a reference image that is usedin example embodiments of the invention.

FIG. 5 is a diagram showing an example of an inspection image that hasundergone image transformation and a reference image in exampleembodiments of the invention.

FIG. 6 is a diagram showing an example of an anomaly certainty factorimage that is obtained in example embodiments of the invention.

FIG. 7 is a flowchart showing operations in a learning mode by theanomaly detection apparatus according to the first example embodiment ofthe invention.

FIG. 8 is a flowchart showing operations in an anomaly detection mode bythe anomaly detection apparatus according to the first exampleembodiment of the invention.

FIG. 9 is a block diagram showing a specific configuration of theanomaly detection apparatus according to a second example embodiment ofthe invention.

FIG. 10 is a flowchart showing operations in a learning mode by theanomaly detection apparatus according to the second example embodimentof the invention.

FIG. 11 is a block diagram showing an example of a computer thatrealizes the anomaly detection apparatus according to the first andsecond example embodiments of the invention.

EXAMPLE EMBODIMENTS First Example Embodiment

Hereinafter, an anomaly detection apparatus, an anomaly detection methodand a program according to a first example embodiment of the inventionwill be described, with reference to FIGS. 1 to 8.

Apparatus Configuration

Initially, a configuration of the anomaly detection apparatus accordingto the first example embodiment will be described using FIG. 1. FIG. 1is a block diagram showing a schematic configuration of the anomalydetection apparatus according to the first example embodiment of theinvention.

An anomaly detection apparatus 100 according to the first exampleembodiment shown in FIG. 1 is for detecting anomalies in an inspectionobject, using an inspection image in which the inspection object appearsand a reference image showing a normal state of the inspection object.As shown in FIG. 1, the anomaly detection apparatus 100 is provided withan image transformation unit 103, an image change detection unit 104, achange detection parameter learning unit 106, and an imagetransformation parameter learning unit 108.

The image transformation unit 103 calculates an image transformationparameter, based on the inspection image, the reference image and aparameter for image transformation parameter calculation, and performsimage transformation on the inspection image, using the calculated imagetransformation parameter, such that the inspection object in theinspection image overlaps with the inspection object in the referenceimage.

The image change detection unit 104 calculates an anomaly certaintyfactor indicating whether there is a change in a specific region of theinspection image, by collating the inspection image that has undergoneimage transformation with the reference image using a change detectionparameter.

The change detection parameter learning unit 106 learns the changedetection parameter, based on the difference between a training imageindicating a correct answer value of the change in the specific regionand the anomaly certainty factor calculated by the image changedetection unit 104.

The image transformation parameter learning unit 108 learns theparameter for image transformation parameter calculation, based on acorrection amount derived from the difference between the training imageand the anomaly certainty factor calculated by the image changedetection unit 104 and to be applied to the inspection image that hasundergone image transformation.

In this way, in the first example embodiment, the anomaly detectionapparatus, through learning using a training image, simultaneouslyoptimizes the parameter (image transformation parameter) for imagetransformation suitable for anomaly detection and the parameter (changedetection parameter) for identifying whether an image that has undergoneimage transformation is anomalous or normal. Thus, according to thefirst example embodiment, highly accurate anomaly detection becomespossible.

Next, the configuration of the anomaly detection apparatus according tothe first example embodiment will be more specifically described usingFIGS. 2 to 6. FIG. 2 is a block diagram showing a specific configurationof the anomaly detection apparatus according to the first exampleembodiment of the invention.

As shown in FIG. 2, in the first example embodiment, the anomalydetection apparatus 100 is further provided with an inspection imageinput unit 101, a reference image input unit 102, a training image inputunit 105 and a correction amount calculation unit 107, in addition tothe configuration shown in FIG. 1.

Also, in the first example embodiment, the anomaly detection apparatusoperates in two operational modes, namely, a learning mode for learningrespective parameters and an anomaly detection mode for detectinganomalies in an inspection object. Therefore, first, the units from theinspection image input unit 101 to the image change detection unit 104that function in both modes will be specifically described.

The inspection image input unit 101 acquires an inspection image fromoutside, and inputs the acquired inspection image to the imagetransformation unit 103. The inspection image is an image obtained byshooting an inspection object with an image capturing apparatus. Also,the inspection image may be a color image, or may be a luminance image,a far-infrared image, or the like. FIG. 3 is a diagram showing anexample of the inspection image that is used in the example embodimentsof the invention. In the example in FIG. 3, the inspection object is acomponent 301 that has a flaw 302. Also, in the example in FIG. 3, theinspection image is a luminance image. Furthermore, in the inspectionimage shown in FIG. 3, the component 301 serving as the inspectionobject has turned a few degrees counterclockwise.

The reference image input unit 102 acquires a reference image fromoutside, and inputs the acquired reference image to the imagetransformation unit 103. The reference image is an image that is usedfor comparison with the inspection image, and, as described above,represents a normal state of the inspection object. Also, the referenceimage is basically the same type of image as the inspection image. FIG.4 is a diagram showing an example of the reference image that is used inthe example embodiments of the invention. In the example in FIG. 4, thecomponent 301 in a normal state appears as the object.

Also, as shown in FIGS. 3 and 4, the position and orientation within theimage may differ between the inspection object that appears in thereference image and the inspection object that appears in the inspectionimage, due to differences in the relative positional relationship withthe camera at the time of shooting the respective images. Differences inpositon and orientation are resolved through image transformation of theinspection image by the image transformation unit 103.

The image transformation unit 103, as described above, performs imagetransformation on the inspection image, such that the inspection objectin the inspection image overlaps with the inspection object in thereference image. Also, as shown in FIG. 2, in the first exampleembodiment, the image transformation unit 103 is broadly provided withan image transformation parameter calculation unit 601 that calculatesthe image transformation parameter and a transformed inspection imagegeneration unit 602 that transforms the inspection image based on theimage transformation parameter calculated by the image transformationparameter calculation unit 601 and generates a transformed inspectionimage.

The image transformation parameter calculation unit 601 calculates theimage transformation parameter, based on the reference image, theinspection image and the parameter for image transformation parametercalculation that is held at the image transformation unit 103.Hereinafter, the image transformation parameter will be specificallydescribed using mathematical equations.

The image transformation parameter is, in the case where scaling isassumed, for example, represented by a set of four parameters, namely,an enlargement factor relating to an x-axis direction, an enlargementfactor relating to a y-axis direction, a center x-coordinate of scaletransformation, and also a center y-coordinate of scale transformation.For example, in the case where affine transformation is assumed, animage transformation parameter θ_(ij) is represented by six parameters{θ₁₁, θ₁₂, θ₁₃, θ₂₁, θ₂₂, θ₂₃} corresponding to the following equation1.

$\begin{matrix}{\begin{pmatrix}x^{\prime} \\y^{\prime}\end{pmatrix} = {\begin{pmatrix}\theta_{11} & \theta_{12} & \theta_{13} \\\theta_{21} & \theta_{22} & \theta_{23}\end{pmatrix}\begin{pmatrix}x \\y\end{pmatrix}}} & \left\lbrack {{Equation}\mspace{14mu} 1} \right\rbrack\end{matrix}$

Also, the image transformation parameter θ_(ij) is calculated by thefollowing equation 2, using a reference image I_(R), an inspection imageI, and a parameter ω₁ for image transformation parameter calculation.θ_(ij) =f _(ij) ¹(I,I _(R),ω₁)  [/Equation 2]

In the above equation 2, a function f_(ij) ¹ is partially differentiablewith respect to ω₁. For example, in the case where the function f_(ij) ¹is constituted by a neural network whose inputs are pixel values of thereference image I_(R) and the inspection image I, a weight parameterthereof may be set to ω₁.

Also, the transformed inspection image generation unit 602 executesimage transformation on the inspection image, based on the calculatedimage transformation parameter, when the image transformation parameterhas been calculated. Here, the transformed inspection image is denotedas a transformed inspection image I_(T). As shown in FIG. 5, theposition and orientation of the component 301 in the inspection imagehave been corrected through image transformation, and are aligned withthe position and orientation of the component 301 in the referenceimage. FIG. 5 is a diagram showing an example of the inspection imagethat has undergone image transformation and the reference image in theexample embodiments of the invention.

Note that, in the first example embodiment, the image transformation bythe transformed inspection image generation unit 602 is nottransformation using a fixed parameter value. A feature of the firstexample embodiment is that the transformation parameter changesdepending on the contents of the reference image I_(R) and theinspection image I.

The image change detection unit 104 calculates an anomaly certaintyfactor, by applying the change detection parameter that is storedtherein to the transformed inspection image that is output by the imagetransformation unit 103 and the reference image that is output by thereference image input unit 102.

The anomaly certainty factor, as described above, indicates whetherthere a change in a specific region of the inspection image. Morespecifically, the anomaly certainty factor is a numerical value D_(k)indicating, for every pixel or every small region constituting theinspection image, the probability that an anomalous change has occurred,and is calculated with the following equation 3. Also, the anomalycertainty factor is a set of the numerical values D_(k) for every pixelor every small region, and is also constituted as an image indicating ananomalous change. Hereinafter, an image constituted by a set of anomalycertainty factors will be denoted as an “anomaly certainty factorimage”.D _(k) =f _(k) ²(I _(T) ,I _(R),ω₂)  [Equation 3]

In the above equation 3, function f_(k2) indicates a function forcalculating the anomaly certainty factor value of a kth pixel in theanomaly certainty factor image, and ω₂ indicates the change detectionparameter. The function f_(k) ² is partially differentiable with respectto the change detection parameter ω₂ and the transformed inspectionimage I_(T). Also, the function f_(k2) may be constituted using a neuralnetwork such as a convolutional neural network, for example.

The anomaly certainty factor image obtained from the reference image andinspection image shown in FIG. 5 is shown in FIG. 6. FIG. 6 is a diagramshowing an example of the anomaly certainty factor image that isobtained in the example embodiments of the invention. In the example inFIG. 6, the portion colored in white indicates a region constituted bypixels whose anomaly certainty factor approximates 0 (minimum value).Also, the portion colored in black indicates a region constituted bypixels whose anomaly certainty factor approximates 1 (maximum value).Furthermore, the portion colored in gray indicates a region constitutedby pixels whose anomaly certainty factor is an intermediate valuebetween 0 and 1. Also, as evident from FIG. 6, the output of the imagechange detection unit 104 is more a set of scalar values holdingposition information than scalar values not including positioninformation. In FIG. 6, reference numeral 501 indicates the portionwhere an anomaly has occurred.

Next, the units from the training image input unit 105 to the imagetransformation parameter learning unit 108 that function only in thelearning mode will be specifically described.

The training image input unit 105 acquires a training image fromoutside, and inputs the acquired training image to the change detectionparameter learning unit 106. The training image is an image indicating acorrect answer value of a change in a specific region as describedabove, and, specifically, is a set of ideal output values for theanomaly certainty factor image (two-dimensional anomaly certainty factorvalues) that is output by the image change detection unit 104. Forexample, in correspondence with the certainty factor image shown in FIG.6, an image in which the value of pixels constituting the entire graphiccorresponding to the anomaly, that is, the inner and contour portions,is “1” and the value of pixels constituting the remaining portion is “0”is given as an example of an ideal training image.

The change detection parameter learning unit 106, in this exampleembodiment, first calculates the sum of squares (denoted as S) of thedifference between the training image input from the training imageinput unit 105 and the anomaly certainty factor image calculated by theimage change detection unit 104. Next, the change detection parameterlearning unit 106 corrects the change detection parameter ω₂ that isstored in the image change detection unit 104, such that the sum ofsquares S becomes smaller than the current state.

As is evident from the abovementioned equation 3, the equation forcalculating S is represented by the change detection parameter ω₂ andthe function D_(k) that is partially differentiable with respect to thetransformed inspection image I_(T) serving as input data. Thus, in thefirst example embodiment, it is possible to correct the change detectionparameter ω₂ and the transformed inspection image I_(T) such that Sdecreases using a steepest descent method. Accordingly, the changedetection parameter learning unit 106 corrects the change detectionparameter with the following equation 4.

$\begin{matrix}\left. \omega_{2}\leftarrow{\omega_{2} - {ɛ_{2}\frac{\partial S}{\partial\omega_{2}}}} \right. & \left\lbrack {{Equation}\mspace{14mu} 4} \right\rbrack\end{matrix}$

Here, ε₂ is a constant that controls the correction amount of onecorrection. The value of the corrected change detection parameter isstored. Such a parameter correction method is also used with a backpropagation method in a convolutional neural network.

The correction amount calculation unit 107 calculates the correctionamount to be applied to the inspection image that has undergone imagetransformation, such that the difference between the training image andthe anomaly certainty factor calculated by the image change detectionunit 104 decreases. Specifically, the correction amount calculation unit107 calculates a correction amount D_(T) of the image-transformedinspection image I_(T) with the following equation 5, such that the sumof squares S of the difference between the training image input from thetraining image input unit 105 and the anomaly certainty factor imagecalculated by the image change detection unit 104 decreases.

$\begin{matrix}{D_{T} = {{- ɛ_{2}}\frac{\partial S}{\partial I_{T}}}} & \left\lbrack {{Equation}\mspace{14mu} 5} \right\rbrack\end{matrix}$

D_(T) is an amount of change that is respectively obtained for eachpixel or some of the pixels of the image-transformed inspection imageI_(T). The correction amount calculation unit 107 outputs the calculatedD_(T) to the image transformation parameter learning unit 108.

The image transformation parameter learning unit 108 corrects andupdates the parameter for image transformation parameter calculationthat is stored in the image transformation unit 103, such that thecorrection amount of the image-transformed inspection image approachesthe correction amount D_(T) input from the correction amount calculationunit 107.

Specifically, the image transformation parameter learning unit 108 firstcalculates the amount of movement of the transformed inspection image inthe x-axis direction and in the y-axis direction so as to realize thecorrection amount (i.e., increase or decrease that depends on the rateby which the pixel values are to be increased or decreased) input fromthe correction amount calculation unit 107. This can be realized byrespectively deriving the differential in the x-axis direction and thedifferential in the y-axis direction on the transformed inspectionimage.

Next, the image transformation parameter learning unit 108 derives theparameter ω₁ for image transformation parameter calculation that iscalculated with respect to various pixel positions and realizes theamount of movement in the x-axis direction and in the y-axis directionof the transformed inspection image by the least-squares method. Asmentioned in the description of the image transformation unit 103, theamount of movement in the x-axis and y-axis directions (x-x′, y-y′) andthe parameter ω₁ for image transformation parameter calculation arerelated. Accordingly, the update amount of the parameter ω₁ for imagetransformation parameter calculation can be readily calculated (refer tothe following reference document). Also, the image transformationparameter learning unit 108 passes the derived parameter ω₁ for imagetransformation parameter calculation to the image transformation unit103 to be stored.

Reference Document: Spatial Transformer Networks, Max Jaderberg, KarenSimonyan, Andrew Zisserman, Koray Kavakcouglu, Equations (7) and (10),https://arxiv.org/abs/1506.02025

Apparatus Operations

Next, operations of the anomaly detection apparatus 100 according to thefirst example embodiment of the invention will be described using FIGS.7 and 8. In the following description, FIGS. 1 to 6 are taken intoconsideration as appropriate. Also, in the first example embodiment, theanomaly detection method is implemented by operating the anomalydetection apparatus 100. Therefore, the following description of theoperations of the anomaly detection apparatus 100 is given in place of adescription of the anomaly detection method according to the firstexample embodiment.

Initially, the case where the anomaly detection apparatus 100 operatesin the learning mode will be described using FIG. 7. FIG. 7 is aflowchart showing operations in the learning mode by the anomalydetection apparatus according to the first example embodiment of theinvention.

As shown in FIG. 7, first, the inspection image input unit 101 acquiresan inspection image for learning, and inputs the acquired inspectionimage to the image transformation unit 103 (step S701). Next, thereference image input unit 102 acquires a reference image for learning,and inputs the acquired reference image to the image transformation unit103 (step S702).

Note that, in the learning mode, the inspection image that is acquiredin step S701 and the reference image that is acquired in step S702 neednot be limited to a specific image, and various images may be used asthese images.

Next, the image transformation unit 103 calculates the transformationparameter, using the inspection image for learning, the reference imagefor learning and the parameter for transformation parameter calculationthat is currently held, and, furthermore, executes image transformationon the inspection image for learning based on the calculatedtransformation parameter (step S703). A transformed inspection image tobe used in learning is thereby generated.

Next, the image change detection unit 104 collates the transformedinspection image generated in step S703 with the reference image, usingthe change detection parameter at the current point in time, andcalculates the anomaly certainty factor for every specific region(specifically, every pixel) (step S704).

Next, the training image input unit 105 acquires a training image, andinputs the acquired training image to the change detection parameterlearning unit 106 (step S705). The training image that is acquired instep S705 is a set of ideal anomaly certainty factor values for everypixel with respect to a group of an inspection image and reference imagefor learning.

Next, the change detection parameter learning unit 106 derives thedifference between the anomaly certainty factor calculated in step S704and the training image (ideal anomaly certainty factor) acquired in stepS705, and learns the change detection parameter, based on the deriveddifference (step S706).

Specifically, the change detection parameter learning unit 106calculates the update amount of the change detection parameter that isstored in the image change detection unit 104, such that the sum ofsquares (square error) of the derived difference becomes smaller thanthe current state, and updates the change detection parameter, using thecalculated update amount.

Next, the correction amount calculation unit 107 derives the differencebetween the anomaly certainty factor calculated in step S704 and thetraining image (ideal anomaly certainty factor) acquired in step S705.Next, the correction amount calculation unit 107 calculates thecorrection amount (amount of change in pixel value) for theimage-transformed inspection image for learning that is input to theimage change detection unit 104, such that the sum of squares (squareerror) of the derived difference decreases below the current state (stepS707). Also, the correction amount calculation unit 107 outputs thecalculated correction amount to the image transformation parameterlearning unit 108.

Next, the image transformation parameter learning unit 108 derives, forevery pixel, the amount of movement in the x-axis direction and in they-axis direction that cause a change in each pixel value in theimage-transformed inspection image for learning. The imagetransformation parameter learning unit 108 then updates and learns theparameter ω₁ for image transformation parameter calculation stored inthe image transformation unit 103, such that the derived amount ofmovement occurs (step S708).

Thereafter, the image transformation parameter learning unit 108determines whether the learning processing has ended (step S709). If theresult of the determination of step S709 indicates that the learningprocessing has not ended, step S701 is executed again. On the otherhand, if the result of the determination of step S709 indicates that thelearning processing has ended, the learning processing ends.

The abovementioned processing from step S701 to step S708 may beexecuted for every group of a training image, a reference image and aninspection image for learning. Also, in accordance with a techniquecalled mini-batching, first, the processing from step S701 to step S704may be executed for group data of a predetermined number of trainingimages, reference images and inspection images for learning. In thiscase, thereafter, a value obtained by accumulating the square error ofthe anomaly certainty factor and the ideal anomaly certainty factor foreach data group may be calculated, and the processing from steps S705 toS708 may be executed once, based on the obtained cumulative value ofsquare errors.

Also, the abovementioned processing from steps S701 to S708 may berepeatedly executed a predetermined number of times, or may berepeatedly executed until the error is less than or equal to athreshold. The parameter for image transformation parameter calculationstored in the image transformation unit 103 and the change detectionparameter stored in the image change detection unit 104 after theprocessing of these steps has been repeatedly executed will be thelearning result that is finally output in the learning mode.

Next, the case where the anomaly detection apparatus 100 operates in theanomaly detection mode using FIG. 8 will be described. FIG. 8 is aflowchart showing operations in the anomaly detection mode by theanomaly detection apparatus according to the first example embodiment ofthe invention.

The anomaly detection mode is executed by the inspection image inputunit 101, the reference image input unit 102, the image transformationunit 103, and the image change detection unit 104. Also, in the anomalydetection mode, the parameter for transformation parameter calculationand the change detection parameter that were generated in the learningmode are used, and anomalous places are detected from the inspectionimage.

As shown in FIG. 8, the inspection image input unit 101 acquires aninspection image, and inputs the acquired inspection image to the imagetransformation unit 103 (step S801). Next, the reference image inputunit 102 acquires a reference image, and inputs the acquired referenceimage to the image transformation unit 103 (step S802).

Next, the image transformation unit 103 calculates the transformationparameter, using the inspection image acquired in step S801, thereference image acquired in step S802, and the parameter fortransformation parameter calculation. The image transformation unit 103then executes image transformation on the inspection image based on thecalculated transformation parameter (step S803). A transformedinspection image is thereby generated.

Next, the image change detection unit 104 collates the transformedinspection image generated in step S803 with the reference imageacquired in step S802, using the change detection parameter, andcalculates an anomaly certainty factor for every specific region(specifically, every pixel) (step S804). Also, the image changedetection unit 104 displays the calculated anomaly certainty factors onthe screen of a display device or the like. The user of the anomalydetection apparatus 100 is thereby able to check whether an anomaly hasoccurred in the inspection object.

Thereafter, the image change detection unit 104 determines whether thedetection processing has ended (step S805). If the result of thedetermination of step S805 indicates that the detection processing hasnot ended, step S801 is executed again. On the other hand, if the resultof the determination of step S805 indicates that the detectionprocessing has ended, the anomaly detection mode ends.

Effects of First Example Embodiment

In this way, change that occurs in the inspection object can besensitively detected, by adopting the configuration described in thefirst example embodiment. This is because the parameter fortransformation parameter calculation in the image transformation unit103 that is located upstream is corrected, such that the differencebetween the output of the image change detection unit 104 and thetraining image decreases, and, as a result, a parameter fortransformation parameter calculation suitable for change detection islearned. That is, according to the first example embodiment, imagetransformation suitable for change detection is realized.

Also, in the abovementioned example, the anomaly certainty factor isdefined as being one for every pixel or every small region, but, in thefirst example embodiment, an anomaly certainty factor defined for everylabel indicating a type of anomaly may be used, instead of theabovementioned anomaly certainty factor. For example, assuming two typesof labels, namely normal and anomalous, two certainty factors aredefined for every pixel or every small region, and, if normal, thecertainty factor for the normal label is set to 1 and the anomalycertainty factor for the anomalous label is set to 0. If anomalous, thecertainty factor for the normal label may be set to 0 and the anomalycertainty factor for the anomalous label may be set to 1. Also, threetypes of labels, namely, normal, anomalous, and beyond consideration,may be assumed.

Program

A program according to the first example embodiment need only be aprogram that causes a computer to execute steps S701 to S709 shown inFIG. 7 and steps S801 to S805 shown in FIG. 8. The anomaly detectionapparatus 100 and the anomaly detection method according to the firstexample embodiment can be realized, by this program being installing ona computer and executed. In this case, a processor of the computerperforms processing, while functioning as the inspection image inputunit 101, the reference image input unit 102, the image transformationunit 103, the image change detection unit 104, the training image inputunit 105, the change detection parameter learning unit 106, thecorrection amount calculation unit 107, and the image transformationparameter learning unit 108.

Also, programs according to this example embodiment may be executed by acomputer system constructed from a plurality of computers. In this case,for example, the computers may respectively function as one of theinspection image input unit 101, the reference image input unit 102, theimage transformation unit 103, the image change detection unit 104, thetraining image input unit 105, the change detection parameter learningunit 106, the correction amount calculation unit 107, and the imagetransformation parameter learning unit 108.

Second Example Embodiment

Next, an anomaly detection apparatus, an anomaly detection method and aprogram according to a second example embodiment of the invention willbe described, with reference to FIGS. 9 and 10.

[Apparatus Configuration]

Initially, a configuration of the anomaly detection apparatus accordingto the second example embodiment will be described using FIG. 9. FIG. 9is a block diagram showing a specific configuration of the anomalydetection apparatus according to the second example embodiment of theinvention.

In the abovementioned first example embodiment, only a label imageindicating a correct answer value of an anomalous region in thecoordinate system of the reference image is used as a training image forlearning, whereas, in the second example embodiment, an ideallytransformed inspection image is, furthermore, also used.

Thus, as shown in FIG. 9, an anomaly detection apparatus 200 accordingto the second example embodiment is provided with an ideally transformedinspection image input unit 201. The ideally transformed inspectionimage input unit 201 acquires an inspection image obtained in the casewhere ideal transformation is performed (hereinafter “ideallytransformed inspection image”) that corresponds to a group of aninspection image and reference image for learning, and inputs thisideally transformed inspection image to the image transformationparameter learning unit 108.

Also, the “ideally transformed inspection image” is an inspection imageobtained through image transformation performed in advance such that theposition and orientation of the inspection object coincides with theposition and orientation of the inspection object in the referenceimage.

Furthermore, in the second example embodiment, the functions of theimage transformation parameter learning unit 108 and the operation inthe learning mode differ from the first example embodiment, due to theabovementioned configuration. Note that the second example embodiment isotherwise similar to the first example embodiment. Hereinafter,differences from the first example embodiment will be described indetail.

The image transformation parameter learning unit 108, in the secondexample embodiment, executes the learning mode in two steps consistingof a first step and a second step. The image transformation parameterlearning unit 108 executes the processing described in the first exampleembodiment in the second step of the learning mode.

On the other hand, in the first step of the learning mode, the imagetransformation parameter learning unit 108 first derives the differencebetween the transformed inspection image that is output by the imagetransformation unit 103 and the ideally transformed inspection imageinput from the ideally transformed inspection image input unit 201.Next, the image transformation parameter learning unit 108 derives theimage transformation parameter, based on the derived difference.

Thereafter, the image transformation parameter learning unit 108 derivesa parameter for image transformation parameter calculation capable ofobtaining the derived image transformation parameter, and stores thederived parameter for image transformation parameter calculation in theimage transformation unit 103.

Also, in the second example embodiment, the ideally transformedinspection image input unit 201 is able to derive the amount of movementalong the x-axis and y-axis from the reference image to the ideallytransformed inspection image for every corresponding point (e.g., everycorresponding pixel) between the ideally transformed inspection imageand the reference image. Note that this movement amount is denoted as afirst movement amount.

Furthermore, in the second example embodiment, the image transformationunit 103, in the learning mode, is able to derive the amount of movementfrom the reference image to the image-transformed inspection image, forevery corresponding point (e.g., every corresponding pixel) between theimage-transformed inspection image and the reference image. Note thatthis movement amount is denoted as a second movement amount.

In this case, the image transformation parameter learning unit 108derives the difference between the first movement amount and the secondmovement amount, for every common corresponding point (every pixel),and, furthermore, updates the parameter for image transformationparameter calculation such that the sum (weighted sum) of deriveddifferences decreases.

Apparatus Operations

Next, the operations of the anomaly detection apparatus 200 according tothe second example embodiment of the invention will be described usingFIG. 10. In the following description, FIG. 9 is taken intoconsideration as appropriate. Also, in the second example embodiment,the anomaly detection method is implemented by operating the anomalydetection apparatus 200. Therefore, the following description of theoperations of the anomaly detection apparatus 200 is given in place of adescription of the anomaly detection method according to the secondexample embodiment.

Also, in the second example embodiment, processing in the anomalydetection mode is similar to the first example embodiment. Thus,hereinafter, the learning mode will be described with reference to FIG.10. FIG. 10 is a flowchart showing operations in the learning mode bythe anomaly detection apparatus according to the second exampleembodiment of the invention.

As shown in FIG. 10, first, in order to execute the first step of thelearning mode, the inspection image input unit 101 acquires aninspection image for learning, and inputs the acquired inspection imageto the image transformation unit 103 (step S901). Next, the referenceimage input unit 102 acquires a reference image for learning, and inputsthe acquired reference image to the image transformation unit 103 (stepS902).

Next, the ideally transformed inspection image input unit 201 acquiresthe ideally transformed inspection image, and inputs the acquiredideally transformed inspection image to the image transformationparameter learning unit 108 (step S903). Also, in step S903, the ideallytransformed inspection image input unit 201, for each pixel constitutingthe ideally transformed inspection image, loads the x-coordinate andy-coordinate of a pixel of the reference image corresponding thereto,and calculates the first movement amount from the reference image to theideally transformed inspection image.

Next, the image transformation unit 103 calculates a transformationparameter, using the inspection image for learning, the reference imagefor learning and the parameter for transformation parameter calculationthat is currently held, and, furthermore, executes image transformationon the inspection image for learning, based on the calculatedtransformation parameter (step S904). Also, in step S904, the imagetransformation unit 103 calculates the second movement amount from thereference image to the transformed inspection image, for each pixel ofthe transformed inspection image.

Next, the image transformation parameter learning unit 108 derives thedifference between the ideally transformed inspection image acquired instep S903 and the image-transformed inspection image generated in stepS904, and updates the parameter for image transformation parametercalculation such that the derived difference decreases (step S905).

Specifically, in step S904, the image transformation parameter learningunit 108 derives the difference between the first movement amount andthe second movement amount for every pixel, calculates an imagetransformation parameter using a back propagation method or the like,such that the derived difference decreases, and updates the parameterfor image transformation parameter calculation, such that the calculatedimage transformation parameter is obtained. Thereafter, the imagetransformation unit 103 stores the updated parameter for imagetransformation parameter calculation.

Next, the image transformation parameter learning unit 108 determineswhether an end condition is met (step S906). Note that the processing ofsteps S901 to S905 being executed a predetermined number of times andthe total of the errors (differences) for all learning patterns beingless than or equal to a predetermined threshold or the like are given asan example of an end condition in this case.

If the result of the determination of step S905 indicates that the endcondition is not met, step S901 is executed again. On the other hand, ifthe result of the determination of step S905 indicates that an endcondition is met, the processing ends. Thereafter, in the anomalydetection apparatus 200, the processing from steps S701 to S708 shown inFIG. 7 in the first example embodiment is executed, as the second stepof the learning mode.

Effects of Second Example Embodiment

In this way, according to the configuration described in the secondexample embodiment, compared with the first example embodiment, learningof the image transformation parameter can be actively advanced in thefirst step of the learning mode, and the image transformation parametercan be learned faster. In addition, after alignment of the inspectionimage and the reference image has been performed comparativelyfavorably, learning of the change detection parameter and adaptivelearning towards image change detection of an image transformationparameter are performed in the second step of the learning mode. Thus,according to the second example embodiment, the time taken in the secondhalf of learning can be shortened, and it becomes possible to acceleratethe learning processing.

Variation 1

In the second example embodiment, a mode may be adopted in which thelearning mode is not divided into two steps. In this mode, learning inthe learning mode is performed end-to-end, similarly to the mode shownin FIG. 7 in the first example embodiment. Also, in this case, thefollowing two types (a) and (b) can be calculated as the difference.

(a) Sum of differences between the x-coordinate and y-coordinate forevery pixel of the image-transformed inspection image that is output bythe image transformation unit 103 and the correct answer values of thex-coordinate and y-coordinate for every pixel of the ideally transformedinspection image.

(b) Sum of square errors between the anomaly certainty factor that iscalculated by the image change detection unit 104 and the ideal anomalycertainty factor that is input as the training image.

Thus, in the second example embodiment, a mode may be adopted in whichthe sums of differences (a) and (b) are weighted, and both the changedetection parameter learning unit 106 and the image transformationparameter learning unit 108 correct the parameters, so as to decreasethe total value of the weighted sums of differences.

Variation 2

Also, in the second example embodiment, two types of systems areconceivable as the anomaly detection mode. One system involves operatingthe ideally transformed inspection image input unit 201 only at the timeof the learning mode, and not performing input from the ideallytransformed inspection image input unit 201 at the time of the anomalydetection mode. According to this system, it is possible to detectanomalies as long as an inspection image and a reference image areinput.

The other system, similarly to the time of the learning mode, involvesthe ideally transformed inspection image from the ideally transformedinspection image input unit 201 being utilized in the computationaloperations of the image transformation unit 103 and the image changedetection unit 104, even at the time of the anomaly detection mode.According to this system, anomalies can be even more reliably detected,because more appropriate image transformation can be performed by theimage transformation unit 103 with the same substance as the first stepof the learning mode.

Variation 3

In the abovementioned example, an inspection image in which pixelpositions of the inspection object have been transformed into pixelpositions of the inspection object in the reference image is used as theideally transformed inspection image. In the second example embodiment,however, the x-coordinate and y-coordinate of each corresponding point(point pair) with the inspection image in the ideally transformedinspection image may be used as the ideally transformed inspectionimage.

Also, in this case, the image transformation parameter learning unit 108derives the difference between the amount of movement of a pixel that isbased on the image transformation parameter output by the imagetransformation parameter calculation unit 601 and the amount of movementof each corresponding point calculated from the x-coordinate andy-coordinate of the corresponding point. The image transformationparameter learning unit 108 then updates and learns the parameter forimage transformation parameter calculation, such that the sum of thedifferences for every corresponding point that is derived decreases.

Also, in the case where the x-coordinate and y-coordinate of eachcorresponding point (point pair) with the inspection image in theideally transformed inspection image are used as the ideally transformedinspection image, the functions of the image transformation unit 103 andthe image transformation parameter learning unit 108 will be as follows.

In the image transformation unit 103, the image transformation parametercalculation unit 601 calculates the x-coordinate and y-coordinate of apredetermined number of feature points from the inspection image and thereference image, respectively. At this time, feature points areextracted in corresponding pairs.

Also, the computational operation for calculating the x-coordinate andy-coordinate of feature points is assumed to partially differentiablewith respect to movement in the x-axis direction and the y-axisdirection. The transformed inspection image generation unit 602 performsimage transformation on the inspection image, using an algorithm fornonlinear image transformation of the control point base such as a thinplate spline method, based on information of the feature points that arepaired, and outputs an image-transformed inspection image.

The image transformation parameter learning unit 108, in the first stepof learning, updates and learns the parameter for image transformationparameter calculation, such that the x-coordinate and the y-coordinateof corresponding point pairs input from the ideally transformedinspection image input unit 201 are respectively output.

Also, the image transformation parameter learning unit 108, in thesecond step of learning, weights the sums of differences (a) and (b),similarly to the abovementioned variation 1, and causes both the changedetection parameter learning unit 106 and the image transformationparameter learning unit 108 to correct the parameters, such that thetotal value of the weighted sums of differences decreases

Program

A program according to the second example embodiment need only be aprogram that causes a computer to execute steps S901 to S906 shown inFIG. 10 and steps S801 to S805 shown in FIG. 8. The anomaly detectionapparatus 200 and the anomaly detection method according to the secondexample embodiment can be realized, by this program being installed on acomputer and executed. In this case, a processor of the computerperforms processing, while functioning as the inspection image inputunit 101, the reference image input unit 102, the image transformationunit 103, the image change detection unit 104, the training image inputunit 105, the change detection parameter learning unit 106, thecorrection amount calculation unit 107, the image transformationparameter learning unit 108, and the ideally transformed inspectionimage input unit 201.

Also, programs according to this example embodiment may be executed by acomputer system constructed from a plurality of computers. In this case,for example, the computers may respectively function as one of theinspection image input unit 101, the reference image input unit 102, theimage transformation unit 103, the image change detection unit 104, thetraining image input unit 105, the change detection parameter learningunit 106, the correction amount calculation unit 107, the imagetransformation parameter learning unit 108, and and the ideallytransformed inspection image input unit 201.

Physical Configuration

Here, a computer that realizes an anomaly detection apparatus byexecuting programs according to the first and second example embodimentswill be described using FIG. 11. FIG. 11 is a block diagram showing anexample of a computer that realizes the anomaly detection apparatusaccording to the first and second example embodiments of the invention.

As shown in FIG. 11, a computer 110 is provided with a CPU 111, a mainmemory 112, a storage device 113, an input interface 114, a displaycontroller 115, a data reader/writer 116, and a communication interface117. These units are connected to each other in a data communicablemanner, via a bus 121.

The CPU 111 implements various computational operations, by expandingprograms (codes) according to this example embodiment stored in thestorage device 113 in the main memory 112, and executing these codes inpredetermined order. The main memory 112, typically, is a volatilestorage device such as a DRAM (Dynamic Random Access Memory). Also,programs according to this example embodiment are provided in a state ofbeing stored on a computer-readable recording medium 120. Note thatprograms according to this example embodiment may be distributed overthe Internet connected via the communication interface 117.

Also, a semiconductor storage device such as a flash memory is given asa specific example of the storage device 113, other than a hard diskdrive. The input interface 114 mediates data transmission between theCPU 111 and input devices 118 such as a keyboard and a mouse. Thedisplay controller 115 is connected to the display device 119 andcontrols display by the display device 119.

The data reader/writer 116 mediates data transmission between the CPU111 and the recording medium 120, and executes readout of programs fromthe recording medium 120 and writing of processing results of thecomputer 110 to the recording medium 120. The communication interface117 mediates data transmission between the CPU 111 and other computers.

Also, a general-purpose semiconductor storage device such as a CF(Compact Flash (registered trademark)) card or an SD (Secure Digital)card, a magnetic storage medium such as a flexible disk, and an opticalstorage medium such as a CD-ROM (Compact Disk Read Only Memory) aregiven as specific examples of the recording medium 120.

Note that the anomaly detection apparatus according to the first andsecond example embodiments is also realizable by using hardwarecorresponding to the respective units, rather than by a computer onwhich programs are installed. Furthermore, the anomaly detectionapparatus may be realized in part by programs, and the remainder may berealized by hardware.

The example embodiments described above can be partially or whollyrealized by supplementary notes 1 to 12 described below, but theinvention is not limited to the following description.

Supplementary Note 1

An anomaly detection apparatus for detecting an anomaly in an inspectionobject, using an inspection image in which the inspection object appearsand a reference image showing a normal state of the inspection object,the apparatus comprising:

an image transformation unit configured to calculate an imagetransformation parameter, based on the inspection image, the referenceimage and a parameter for image transformation parameter calculation,and perform image transformation on the inspection image, using thecalculated image transformation parameter, such that the inspectionobject in the inspection image overlaps with the inspection object inthe reference image;

an image change detection unit configured to collate the reference imageand the inspection image that has undergone image transformation, usinga change detection parameter, and calculate an anomaly certainty factorindicating whether there is a change in a specific region of theinspection image;

a change detection parameter learning unit configured to learn thechange detection parameter, based on a difference between a trainingimage indicating a correct answer value of the change in the specificregion and the anomaly certainty factor calculated by the image changedetection unit; and

an image transformation parameter learning unit configured to learn theparameter for image transformation parameter calculation, based on acorrection amount derived from the difference between the training imageand the anomaly certainty factor calculated by the image changedetection unit and to be applied to the inspection image that hasundergone image transformation.

Supplementary Note 2

The anomaly detection apparatus according to supplementary note 1,

in which the change detection parameter learning unit derives thedifference between the training image and the anomaly certainty factor,and learns the change detection parameter by updating a value of thechange detection parameter such that the derived difference decreases,

the anomaly detection apparatus further includes a correction amountcalculation unit configured to calculate the correction amount to beapplied to the inspection image that has undergone image transformation,such that the difference between the training image and the anomalycertainty factor calculated by the image change detection unitdecreases, and

the image transformation parameter learning unit learns the parameterfor image transformation parameter calculation, by updating theparameter for image transformation parameter calculation, such thatimage transformation that produces the correction amount calculated bythe correction amount calculation unit is performed.

Supplementary note 3

The anomaly detection apparatus according to supplementary note 2,further including:

an input unit configured to input an inspection image created in advanceand obtained in a case where the image transformation is ideallyperformed to the image transformation parameter learning unit,

in which the image transformation parameter learning unit derives adifference between the inspection image obtained in the case where theideal image transformation is performed and the inspection image thathas undergone image transformation by the image transformation unit, andupdates the parameter for image transformation parameter calculation,such that the derived difference decreases.

Supplementary Note 4

The anomaly detection apparatus according to supplementary note 3,

in which the input unit derives, for every corresponding point betweenthe inspection image obtained in the case where the ideal imagetransformation is performed and the reference image, a first movementamount from the reference image to the inspection image obtained in thecase where the ideal image transformation is performed,

the image transformation unit derives, for every corresponding pointbetween the image-transformed inspection image and the reference image,a second movement amount from the reference image to theimage-transformed inspection image, and

the image transformation parameter learning unit derives a differencebetween the first movement amount and the second movement amount forevery common corresponding point, and updates the parameter for imagetransformation parameter calculation such that a sum of the deriveddifferences decreases.

Supplementary note 5

An anomaly detection method for detecting an anomaly in an inspectionobject, using an inspection image in which the inspection object appearsand a reference image showing a normal state of the inspection object,the method including:

(a) a step of calculating an image transformation parameter, based onthe inspection image, the reference image and a parameter for imagetransformation parameter calculation, and performing imagetransformation on the inspection image, using the calculated imagetransformation parameter, such that the inspection object in theinspection image overlaps with the inspection object in the referenceimage;

(b) a step of collating the reference image and the inspection imagethat has undergone image transformation, using a change detectionparameter, and calculating an anomaly certainty factor indicatingwhether there is a change in a specific region of the inspection image;

(c) a step of learning the change detection parameter, based on adifference between a training image indicating a correct answer value ofthe change in the specific region and the anomaly certainty factorcalculated in the (b) step; and

(d) a step of learning the parameter for image transformation parametercalculation, based on a correction amount derived from the differencebetween the training image and the anomaly certainty factor calculatedin the (b) step and to be applied to the inspection image that hasundergone image transformation.

Supplementary Note 6

The anomaly detection method according to supplementary note 5,

in which, in the (c) step, the difference between the training image andthe anomaly certainty factor is derived, and the change detectionparameter is learned by updating a value of the change detectionparameter such that the derived difference decreases,

the anomaly detection method further includes (e) a step of calculatingthe correction amount to be applied to the inspection image that hasundergone image transformation, such that the difference between thetraining image and the anomaly certainty factor calculated in the (b)step decreases, and

in the (d) step, the parameter for image transformation parametercalculation is learned, by updating the parameter for imagetransformation parameter calculation, such that image transformationthat produces the correction amount calculated in the (e) step isperformed.

Supplementary note 7

The anomaly detection method according to supplementary note 6, furtherincluding:

(f) a step of inputting an inspection image created in advance andobtained in a case where the image transformation is ideally performed;and

(g) a step of deriving a difference between the inspection imageobtained in the case where the ideal image transformation is performedand the inspection image that has undergone image transformation in the(a) step, and updating the parameter for image transformation parametercalculation, such that the derived difference decreases.

Supplementary Note 8

The anomaly detection method according to supplementary note 7, furtherincluding:

(h) a step of deriving, for every corresponding point between theinspection image obtained in the case where the ideal imagetransformation is performed and the reference image, a first movementamount from the reference image to the inspection image obtained in thecase where the ideal image transformation is performed; and

(i) a step of deriving, for every corresponding point between theimage-transformed inspection image and the reference image, a secondmovement amount from the reference image to the image-transformedinspection image,

in which, in the (g) step, a difference between the first movementamount and the second movement amount is derived for every commoncorresponding point, and the parameter for image transformationparameter calculation is updated such that a sum of the deriveddifferences decreases.

Supplementary note 9

A computer-readable recording medium that includes a program recordedthereon for detecting, by computer, an anomaly in an inspection object,using an inspection image in which the inspection object appears and areference image showing a normal state of the inspection object, theprogram including instructions that cause the computer to carry out:

(a) a step of calculating an image transformation parameter, based onthe inspection image, the reference image and a parameter for imagetransformation parameter calculation, and performing imagetransformation on the inspection image, using the calculated imagetransformation parameter, such that the inspection object in theinspection image overlaps with the inspection object in the referenceimage;

(b) a step of collating the reference image and the inspection imagethat has undergone image transformation, using a change detectionparameter, and calculating an anomaly certainty factor indicatingwhether there is a change in a specific region of the inspection image;

(c) a step of learning the change detection parameter, based on adifference between a training image indicating a correct answer value ofthe change in the specific region and the anomaly certainty factorcalculated in the (b) step; and

(d) a step of learning the parameter for image transformation parametercalculation, based on a correction amount derived from the differencebetween the training image and the anomaly certainty factor calculatedin the (b) step and to be applied to the inspection image that hasundergone image transformation.

Supplementary Note 10

The computer-readable recording medium according to supplementary note9,

in which, in the (c) step, the difference between the training image andthe anomaly certainty factor is derived, and the change detectionparameter is learned by updating a value of the change detectionparameter such that the derived difference decreases,

the program further includes instructions that cause the computer tocarry out (e) a step of calculating the correction amount to be appliedto the inspection image that has undergone image transformation, suchthat the difference between the training image and the anomaly certaintyfactor calculated in the (b) step decreases, and

in the (d) step, the parameter for image transformation parametercalculation is learned, by updating the parameter for imagetransformation parameter calculation, such that image transformationthat produces the correction amount calculated in the (e) step isperformed.

Supplementary note 11

The computer-readable recording medium according to supplementary note10, in which the program further includes instructions that cause thecomputer to carry out:

(f) a step of inputting an inspection image created in advance andobtained in a case where the image transformation is ideally performed;and

(g) a step of deriving a difference between the inspection imageobtained in the case where the ideal image transformation is performedand the inspection image that has undergone image transformation in the(a) step, and updating the parameter for image transformation parametercalculation, such that the derived difference decreases.

Supplementary Note 12

The computer-readable recording medium according to supplementary note11,

in which the program further includes instructions that cause thecomputer to carry out:

(h) a step of deriving, for every corresponding point between theinspection image obtained in the case where the ideal imagetransformation is performed and the reference image, a first movementamount from the reference image to the inspection image obtained in thecase where the ideal image transformation is performed; and

(i) a step of deriving, for every corresponding point between theimage-transformed inspection image and the reference image, a secondmovement amount from the reference image to the image-transformedinspection image,

in which, in the (g) step, a difference between the first movementamount and the second movement amount is derived for every commoncorresponding point, and the parameter for image transformationparameter calculation is updated such that a sum of the deriveddifferences decreases.

Although the invention of the present application has been describedabove with reference to example embodiments, the invention is notlimited to the example embodiments described above. Variousmodifications apparent to those skilled in the art can be made to theconfiguration and details of the invention within the scope of theinvention.

INDUSTRIAL APPLICABILITY

According to the invention as described above, highly accurate anomalydetection becomes possible by simultaneously optimizing a parameter forimage transformation suitable for anomaly detection and a parameter foridentifying whether an image that has undergone image transformation isanomalous or normal. The invention is useful in fields in which anomalydetection using images is required, such as the field of qualitycontrol, for example.

LIST OF REFERENCE SIGNS

-   100 Anomaly detection apparatus-   101 Inspection image input unit-   102 Reference image input unit-   103 Image transformation unit-   104 Image change detection unit-   105 Training image input unit-   106 Change detection parameter learning unit-   107 Correction amount calculation unit-   108 Image transformation parameter learning unit-   110 Computer-   111 CPU-   112 Main memory-   113 Storage device-   114 Input interface-   115 Display controller-   116 Data reader/writer-   117 Communication interface-   118 Input device-   119 Display device-   120 Recording medium-   121 Bus-   200 Anomaly detection apparatus (second example embodiment)-   201 Ideally transformed inspection image input unit-   301 Component-   302 Flaw-   501 Portion where anomaly has occurred-   601 Image transformation parameter calculation unit-   602 Transformed inspection image generation unit

The invention claimed is:
 1. An anomaly detection apparatus fordetecting an anomaly in an inspection object, using an inspection imagein which the inspection object appears and a reference image showing anormal state of the inspection object, the apparatus comprising: animage transformation unit configured to calculate an imagetransformation parameter, based on the inspection image, the referenceimage and a parameter for image transformation parameter calculation,and perform image transformation on the inspection image, using thecalculated image transformation parameter, such that the inspectionobject in the inspection image overlaps with the inspection object inthe reference image; an image change detection unit configured tocollate the reference image and the inspection image that has undergoneimage transformation, using a change detection parameter, and calculatean anomaly certainty factor indicating whether there is a change in aspecific region of the inspection image; a change detection parameterlearning unit configured to learn the change detection parameter, basedon a difference between a training image indicating a correct answervalue of the change in the specific region and the anomaly certaintyfactor calculated by the image change detection unit; and an imagetransformation parameter learning unit configured to learn the parameterfor image transformation parameter calculation, based on a correctionamount derived from the difference between the training image and theanomaly certainty factor calculated by the image change detection unitand to be applied to the inspection image that has undergone imagetransformation.
 2. The anomaly detection apparatus according to claim 1,wherein the change detection parameter learning unit derives thedifference between the training image and the anomaly certainty factor,and learns the change detection parameter by updating a value of thechange detection parameter such that the derived difference decreases,the anomaly detection apparatus further comprises a correction amountcalculation unit configured to calculate the correction amount to beapplied to the inspection image that has undergone image transformation,such that the difference between the training image and the anomalycertainty factor calculated by the image change detection unitdecreases, and the image transformation parameter learning unit learnsthe parameter for image transformation parameter calculation, byupdating the parameter for image transformation parameter calculation,such that image transformation that produces the correction amountcalculated by the correction amount calculation unit is performed. 3.The anomaly detection apparatus according to claim 2, furthercomprising: an input unit configured to input an inspection imagecreated in advance and obtained in a case where the image transformationis ideally performed to the image transformation parameter learningunit, wherein the image transformation parameter learning unit derives adifference between the inspection image obtained in the case where theideal image transformation is performed and the inspection image thathas undergone image transformation by the image transformation unit, andupdates the parameter for image transformation parameter calculation,such that the derived difference decreases.
 4. The anomaly detectionapparatus according to claim 3, wherein the input unit derives, forevery corresponding point between the inspection image obtained in thecase where the ideal image transformation is performed and the referenceimage, a first movement amount from the reference image to theinspection image obtained in the case where the ideal imagetransformation is performed, the image transformation unit derives, forevery corresponding point between the image-transformed inspection imageand the reference image, a second movement amount from the referenceimage to the image-transformed inspection image, and the imagetransformation parameter learning unit derives a difference between thefirst movement amount and the second movement amount for every commoncorresponding point, and updates the parameter for image transformationparameter calculation such that a sum of the derived differencesdecreases.
 5. An anomaly detection method for detecting an anomaly in aninspection object, using an inspection image in which the inspectionobject appears and a reference image showing a normal state of theinspection object, the method comprising: a calculating an imagetransformation parameter, based on the inspection image, the referenceimage and a parameter for image transformation parameter calculation,and performing image transformation on the inspection image, using thecalculated image transformation parameter, such that the inspectionobject in the inspection image overlaps with the inspection object inthe reference image; a collating the reference image and the inspectionimage that has undergone image transformation, using a change detectionparameter, and calculating an anomaly certainty factor indicatingwhether there is a change in a specific region of the inspection image;a learning the change detection parameter, based on a difference betweena training image indicating a correct answer value of the change in thespecific region and the anomaly certainty factor calculated in thecollating; and a learning the parameter for image transformationparameter calculation, based on a correction amount derived from thedifference between the training image and the anomaly certainty factorcalculated in the (b) step and to be applied to the inspection imagethat has undergone image transformation.
 6. A non-transitorycomputer-readable recording medium that includes a program recordedthereon for detecting, by computer, an anomaly in an inspection object,using an inspection image in which the inspection object appears and areference image showing a normal state of the inspection object, theprogram including instructions that cause the computer to carry out: acalculating an image transformation parameter, based on the inspectionimage, the reference image and a parameter for image transformationparameter calculation, and performing image transformation on theinspection image, using the calculated image transformation parameter,such that the inspection object in the inspection image overlaps withthe inspection object in the reference image; a collating the referenceimage and the inspection image that has undergone image transformation,using a change detection parameter, and calculating an anomaly certaintyfactor indicating whether there is a change in a specific region of theinspection image; a learning the change detection parameter, based on adifference between a training image indicating a correct answer value ofthe change in the specific region and the anomaly certainty factorcalculated in the collating; and a learning the parameter for imagetransformation parameter calculation, based on a correction amountderived from the difference between the training image and the anomalycertainty factor calculated in the (b) step and to be applied to theinspection image that has undergone image transformation.
 7. The anomalydetection method according to claim 5, in which, in the learning, thedifference between the training image and the anomaly certainty factoris derived, and the change detection parameter is learned by updating avalue of the change detection parameter such that the derived differencedecreases, the anomaly detection method further includes a calculatingthe correction amount to be applied to the inspection image that hasundergone image transformation, such that the difference between thetraining image and the anomaly certainty factor calculated in thecollating decreases, and in the learning the parameter for imagetransformation parameter calculation, the parameter for imagetransformation parameter calculation is learned, by updating theparameter for image transformation parameter calculation, such thatimage transformation that produces the correction amount calculated inthe calculating is performed.
 8. The anomaly detection method accordingto claim 7, further including: an inputting an inspection image createdin advance and obtained in a case where the image transformation isideally performed, and a deriving a difference between the inspectionimage obtained in the case where the ideal image transformation isperformed and the inspection image that has undergone imagetransformation in the calculating an image transformation parameter, andupdating the parameter for image transformation parameter calculation,such that the derived difference decreases.
 9. The anomaly detectionmethod according to claim 8, further including: a deriving, for everycorresponding point between the inspection image obtained in the casewhere the ideal image transformation is performed and the referenceimage, a first movement amount from the reference image to theinspection image obtained in the case where the ideal imagetransformation is performed; and a deriving, for every correspondingpoint between the image-transformed inspection image and the referenceimage, a second movement amount from the reference image to theimage-transformed inspection image, in which, in the deriving thedifference, a difference between the first movement amount and thesecond movement amount is derived for every common corresponding point,and the parameter for image transformation parameter calculation isupdated such that a sum of the derived differences decreases.
 10. Thenon-transitory computer-readable recording medium according to claim 6,in which, in the learning the change detection parameter, the differencebetween the training image and the anomaly certainty factor is derived,and the change detection parameter is learned by updating a value of thechange detection parameter such that the derived difference decreases,the program further includes instructions that cause the computer tocarry out (e) a step of calculating the correction amount to be appliedto the inspection image that has undergone image transformation, suchthat the difference between the training image and the anomaly certaintyfactor calculated in the collating decreases, and in the learning theparameter for image transformation parameter calculation, the parameterfor image transformation parameter calculation is learned, by updatingthe parameter for image transformation parameter calculation, such thatimage transformation that produces the correction amount calculated inthe calculating is performed.
 11. The non-transitory computer-readablerecording medium according to claim 10, in which the program furtherincludes instructions that cause the computer to carry out: an inputtingan inspection image created in advance and obtained in a case where theimage transformation is ideally performed, and a deriving a differencebetween the inspection image obtained in the case where the ideal imagetransformation is performed and the inspection image that has undergoneimage transformation in the calculating an image transformationparameter, and updating the parameter for image transformation parametercalculation, such that the derived difference decreases.
 12. Thenon-transitory computer-readable recording medium according to claim 11,in which the program further includes instructions that cause thecomputer to carry out: a deriving, for every corresponding point betweenthe inspection image obtained in the case where the ideal imagetransformation is performed and the reference image, a first movementamount from the reference image to the inspection image obtained in thecase where the ideal image transformation is performed; and a deriving,for every corresponding point between the image-transformed inspectionimage and the reference image, a second movement amount from thereference image to the image-transformed inspection image, in which, inthe deriving the difference, a difference between the first movementamount and the second movement amount is derived for every commoncorresponding point, and the parameter for image transformationparameter calculation is updated such that a sum of the deriveddifferences decreases.